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(54) ALL-AROUND VIDEO OUTPUT METHOD AND DEVICE 



(57) An omnidirectional video apparatus that oper- 
ates when an image at an arbitrary view point is cut out 
from an omnidirectional image for reproduction, to min- 
imize the amount of information transferred if all direc- 
tions are stored in another apparatus is provided. Only 
positional information from an omnidirectional image 
position storage means (3) is displayed on an image 
pickup position display means (4) of a terminal device 



via a bidirectional communication means (10). A view 
point position indicating means (5) specifies a required 
view point position. An image converting means (6) of 
a supply center extracts required images from the om- 
nidirectional image position storage means (3) depend- 
ing on the view point position indicated by the view point 
position indicating means (5) and transmits the images 
via the bidirectional communication means (10) to the 
terminal device for display 
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Description 

Technical Field 

[0001] The present invention relates to an apparatus 
for processing panoramic images or the like which have 
been photographed in all directions. 

Background Art 

[0002] It is strongly desired to observe surrounding 
environments at a certain view point as if the viewer was 
at that site. Specifically, if a viewing direction for an im- 
age of scenery photographed during a trip can be freely 
varied to display scenery extending in the viewing direc- 
tion, the viewer can feel as if he or she was looking 
around though he or she is not actually at that site. 
[0003] If the viewing direction can be freely varied for 
an image such as that obtained using a fisheye lens or 
an omnidirectional image obtained using a reflection 
mirror, which image has a field of view much larger than 
that obtained by a normal camera lens, the feeling of 
being at the site is further improved. 
[0004] Japanese Patent Laid-Open No. 9-62861 dis- 
closes an apparatus for storing panoramic images pho- 
tographed around a view point moving while traveling 
through space as well as a relative- moving position from 
a certain point. 

[0005] In reproducing an Image, an image extending 
in a specified viewing direction is synthesized from the 
panoramic image stored in the apparatus, and the syn- 
thesized image is displayed. When the user indicates a 
next view point or line of sight, a next image is corre- 
spondingly invoked, computed, and displayed; this op- 
eration is repeated. 

[0006] The view point is a position where an image is 
photographed, and the line of sight is a direction in which 
a viewer looks at the view point position. 
[0007] FIG. 21 shows a process loop of image repro- 
duction according to this conventional technique. 
[0008] The following description follows this process 
loop. 

[0009] S201 is a loop in which an image is repeatedly 
synthesized and displayed. 

[0010] At S202, a view point and a line-of-sight direc- 
tion for a traveling path are initially computed. After the 
start loop, the next view point position and line-of-sight 
direction to be displayed are computed and determined 
from a relative movement computed at S206, relative to 
the current view point and llne-of-sight direction. 
[001 1 ] At S203, a panoramic image corresponding to 
the view point position determined at S202 is loaded. 
[0012] At S204, an image extending in the line-of- 
sight direction determined at S202 is synthesized from 
the panoramic image. 

[001 3] At S205, the image synthesized at S204 is dis- 
played. 

[0014] At S206, a user views the image displayed at 



S205 to indicate movement of the view point and the 
line-of-sight direction in which a visual angle is moved. 
[001 5] At S207, the process returns to S201 to repeat 
processing from S202 to S206. 
5 [0016] At S206 the user can command move forward, 
move backward, stop, clockwise rotation, counterclock- 
wise rotation, upward rotation, downward rotation, en- 
largement, contraction, and end. 
[0017] Additionally, by retaining history of the view 
10 point position and line-of-sight direction, this conven- 
tional apparatus can predict the next possible view point 
position and line-of-sight direction to be indicated by the 
user in order to load a required image in advance, there- 
by achieving fast display. 
15 [0018] In this manner, for a video photographed in all 
directions during movement, the conventional appara- 
tus repeatedly invokes, for each reproduction, images 
photographed at a view point position and in a line-of- 
sight direction dependent on the user's command and 
synthesizes and displays the images, to provide the us- 
er with images that provide him or her with the feeling 
of being at the site. 

[0019] The above described conventional apparatus, 
however, still has the following problems: 

<1> An image photographed while traveling 
through space is retained, and the user checks the 
reproduced image, indicates relative view point and 
llne-of-sight direction for the image to obtain a re- 
sult, and then synthesizes the next image to be dis- 
played. Thus, after the information on the view point 
and the line-of-sight direction has been provided, a 
required Image is invoked from all stored images 
and then displayed after the image synthesis proc- 
ess. Consequently, the reproduction Is delayed. 

In particular, when panoramic images and rel- 
ative-moving positions are stored at another site on 
the Internet and a communication means that is rel- 
atively slow in communicating data is used for re- 
production, the reproduction is more significantly 
delayed. 

<2> When the next possible view point position and 
line-of-sight direction to be indicated by the user are 
predicted by retaining the history of the view point 
position and line-of-sight direction for the displayed 
image, the reproduction speed increases to some 
degree. When, however, the view point position is 
moved in an unpredictable direction, new invoca- 
tion of an image and new computations therefor are 
required to thereby reduce the reproduction speed 
to prevent the smooth passage of time for the re- 
produced image, thereby degrading the feeling of 
being at the site. 

<3> A position to wh ich the viewpoint is to be moved 
is specified based on a relative position along the 
traveling path, so that the image can only be repeat- 
edly moved forward and backward along the se- 
quence of images photographed while traveling 
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through space. Accordingly, the display of an image 
on an unwanted path cannot be deleted. 
<4> The position to which the view point is to be 
moved is specified based on a relative position 
along the traveling path , so that user does not know 
which way to take next time in an image photo- 
graphed at a site visited by the user for the first time. 
As a result, the user must reach the target by re- 
peating moving forward and backward as required. 

Disclosure of Invention 

[0020] It is an object of the present invention to pro- 
vide an omnidirectional video output method that ena- 
bles fast reproduction and provides users with the feel- 
ing of being at the site. 

[0021] It is another object of the present invention to 
provide an omnidirectional video output method that al- 
lows a user to easily input desired traveling path and 
line of sight to a terminal device to receive display of 
only images on a required path. 

[0022] An omnidirectional video output method ac- 
cording to the present invention comprises transmitting 
image pickup position data representing a service area 
having omnidirectional images, from a supply center to 
a terminal device, and transmitting at least view point 
information of desired view point information and line- 
of-sight information from the terminal device to the sup- 
ply center. When only the view point information is trans- 
mitted from the terminal device to the supply center, only 
image information for a traveling path desired by the ter- 
minal device is transmitted from the supply center to the 
terminal device, which then processes and outputs the 
information depending on the desired view point. When 
the view point and line-of-sight information is transmit- 
ted from the terminal device to the supply center, only 
image information for a traveling path desired by the ter- 
minal device is processed depending on the desired line 
of sight based on the view point and line-of-sight infor- 
mation received from the terminal device and is then 
transmitted from the supply center to the terminal de- 
vice, which then outputs the information. 
[0023] According to the configuration of the present 
invention, the amount of data in information between the 
supply center and the terminal device is reduced to in- 
crease the reproduction speed, thereby providing users 
with the feeling of being at the site. 
[0024] This configuration also improves the input 
method for the terminal device to facilitate the operation 
of inputting the desired traveling path and line of sight. 
[0025] In addition, when the terminal device indicates 
the desired view point to the supply center, the supply 
center presents image pickup position data for omnidi- 
rectional images to the terminal device, which receives, 
based on the image pickup position data, image infor- 
mation for a desired traveling path determined depend- 
ing on the desired view point. Accordingly, the user can 
easily Input the desired traveling path and can continu- 



ously view only images on the desired traveling path in 
such a manner that the reproduction time passes 
smoothly. 

[0026] Further, if not only the view point information 
s but also the line-of-sight information is transmitted from 
the terminal device to the supply center, the supply cent- 
er extracts omnidirectional images from an omnidirec- 
tional image position file based on the view point infor- 
mation, subsequently processes the images based on 
io the line-of-sight information, and then transmits only the 
required image to the terminal device for display. Con- 
sequently, the user can enjoy an all-image display that 
provides users with the feeling of being at the site, re- 
gardless of an arithmetic-operation process speed of 
is the terminal device. 

Brief Description of Drawings 
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FIG. 1 is a block diagram of an omnidirectional vid- 
eo apparatus according to (Embodiment 1) of the 
present invention; 

FIG. 2 is a schematic view of the system configura- 
tion of the omnidirectional video apparatus accord- 
ing to this embodiment; 

FIG. 3 is a view useful in explaining the principle of 
image pickup of omnidirectional images according 
to this embodiment; 

FIG. 4 is a schematic view of omnidirectional imag- 
es according to this embodiment; 
FIG. 5 is a view useful in explaining a traveling path 
for a video sensor according to this embodiment; 
FIG. 6 is a view useful in explaining an omnidirec- 
tional image position file according to this embodi- 
ment; 

FIG. 7 is a view useful in explaining a result of dis- 
play of all image pickup positions according to this 
embodiment; 

FIG. 8 is a view useful in explaining a result of indi- 
cation of an observation path; 
FIG. 9 is a schematic view of a expanded image 
according to this embodiment; 
FIG. 10 is a view useful in explaining a expanded 
image file according to this embodiment; 
FIG. 11 is a view useful in explaining a window im- 
age according to this embodiment; 
FIG. 12 is a view useful in explaining a line-of-sight 
indicating means according to this embodiment; 
FIG. 1 3 is a block diagram of an omnidirectional vid- 
eo apparatus according to (Embodiment 2) of the 
present invention; 

FIG. 14 is a view useful in explaining a result of in- 
dication of an observation path; 
FIG . 1 5 is a block diagram of an omnidirectional vid- 
eo apparatus according to (Embodiment 3) of the 
present invention; 

FIG. 1 6 is a view useful in explaining a result of dis- 
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play of alt Image pickup positions according to this 
embodiment; 

FIG. 17 is a view useful in explaining a result of in- 
dication of an observation path; 
FIG. 18 is a block diagram of an omnidirectional vid- 
eo apparatus according to (Embodiment 4) of the 
present invention; 

FIG. 19 is a view useful in explaining a result of in- 
dication of an object according to this embodiment; 
FIG. 20 is a view useful in explaining a result of com- 
putation for an observation path according this em- 
bodiment; and 

FIG. 21 is a block diagram showing an operational 
procedure executed by a conventional apparatus. 

Best Mode for Carrying Out the Invention 

[0028] Each embodiment of the present invention will 
be explained below with reference to FIGS. 1 to 20. 

(Embodiment 1) 

[0029] FIGS. 1 to 12 show (Embodiment 1). 

[003O] FIG. 1 is an omnidirectional video apparatus 

according to Embodiment 1 . 

[0031] An omnidirectional image pickup means 1 pho- 
tographs omnidirectional images at a view point position 
to output image data. An image pickup position obtain- 
ing means 2 obtains positions where the omnidirectional 
Image pickup means 1 picked up the images. The im- 
ages picked up by the omnidirectional image pickup 
means 1 and the image pickup positions obtained by the 
image pickup position obtaining means 2 are stored in 
an omnidirectional image position storage means 3. 
[0032] The omnidirectional image position storage 
means 3 and an image pickup position display means 4 
are connected together via a bidirectional communica- 
tion means 10 so that image pickup positions stored in 
the omnidirectional image position storage means 3 are 
displayed on the image pickup position display means 4. 
[0033] A user specifies one of the image pickup posi- 
tions displayed by the image pickup position display 
means 4, as his or her desired view point position, by 
inputting it to a view point position indicating means 5. 
The view point position Indicating means 5 is connected 
to an image converting means 6 via a bidirectional com- 
munication means 1 0, and the image converting means 
6 Invokes from the omnidirectional view point position 
storage means 3 an image recorded at a position indi- 
cated by the view point position indicating means 5 and 
geometrically converts and rearranges the image, which 
was photographed in all directions. 
[0034] The image converting means 6 is connected 
to an image processing means 8 via the bidirectional 
communication means 10, and the Image processing 
means 8 has a Hne-of-sight indicating means 7 connect- 
ed thereto for indicating a line-of-sight direction and an 
angle of view for the image obtained from the image con- 



6 

verting means 6. 

[0035] The image processing means 8 cuts a partial 
image out from the image obtained from the image con- 
verting means 6 depending on the line-of -sight direction 
5 and angle of view specified by the llne-of-slgnt indicating 
means 7. The partial image obtained by the image 
processing means 8 by cutting out the original image is 
displayed by the image display means 9. 
[0036] Specifically, the omnidirectional image pickup 
10 means 1 and the image pickup position obtaining means 
2 are provided in a video sensor device 11 in an arrange- 
ment drawing of the entire system shown in FIG. 2. The 
omnidirectional image position storage means 3 and the 
. image converting means 6 are provided in a supply cent- 
's er 12. The image pickup position display means 4, the 
view point position indicating means S, the line-of-sight 
indicating means 7, the image processing means 8, and 
the image display means 9 are provided in a terminal 
device 1 3. The supply center 1 2 and the terminal device 
20 1 3 are connected together via the bidirectional commu- 
nication means 10 for bidirectional information commu- 
nication. 

[0037] Although FIG. 2 shows the one supply center 
12 and the one terminal device 13, a plurality of supply 

25 centers or terminal devices may be connected to the bi- 
directional communication means 10. 
[0038] The video sensor device 11 is configured to 
move in a direction perpendicular to the direction of an 
optical axis of a television camera 15 while mounting a 

30 mirror 14 and the television camera 1 5 on a traveling 
device 17. In this explanation, the traveling device 17 
moves on a horizontal plane. 

[0039] The television camera 1 5 is Installed in such a 
manner that the optical axis extends in a vertical direc- 
35 tion, to pick up an image reflected by the mirror 1 4. The 
mirror 14 is installed in such a manner that the optical 
axis of the television camera 15 aligns with the center 
axis of the mirror 14 so that the image reflected by the 
mirror 1 4 can be picked up to obtain an omnidirectional 
image around the optical axis of the television camera 
15 at one time. 

[0040] Specifically, the mirror 14 is a curved surface 
that is symmetrical with respect to the axis. It may be 
shaped into a cone or a spherical surface but will be de* 

45 scribed as having a hyperboloid. Reference numeral 
1 4a denotes a cylindrical translucent body for support- 
ing the mirror 1 4 at a target position. In this embodiment, 
however, a method for picking up an image extending 
around a view point position through 360° using the mir- 

50 ror 14, but for example, a fisheye lens may be used for 
image pickup or the camera may be rotated to synthe- 
size picked-up images, as long as the image can be ar- 
ranged around the view point position through 360 Q . 
[0041] Additionally, in the following description, the 

55 video sensor device 11 is used outdoors. 

[0042] During outdoor image pickup, brightness is not 
uniform in all directions, so that the user often fails to 
pick up appropriate omnidirectional images with a fixed 
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exposure. Thus, the television camera 1 5 can desirably 
obtain dynamic-range images that are larger than those 
obtained by normal cameras; a preferable television 
camera can synthesize images obtained with a normal 
exposure and a short-time exposure, into one field, 
[0043] The video sensor device 1 1 has a GPS device 
16 mounted thereon as the image pickup position ob- 
taining means 2, to obtain global positions of the video 
sensor device 11 . The GPS device 16 obtains its own 
position on the earth (latitude and longitude) via a sat- 
ellite (not shown). A preferable GPS is a differential one 
that can obtain its own position by detecting its relative 
position to a certain point the latitude and longitude of 
which are known. 

[0044] First, an omnidirectional image picked up by 
the video sensor device 11 will be explained with refer- 
ence to FIGS. 3 and 4. 

[0045] FIG. 3 is a view useful in explaining the princi- 
ple of image pickup of an omnidirectional image. Refer- 
ence numeral 18 denotes a photographing object. Ref- 
erence numeral 1 9 denotes an image forming plane of 
the television camera 1 5. Reference numeral 20 is a fo- 
cal point of the television camera 15. Reference numeral 
21 denotes a position on the image forming plane 19 
where the photographing object 1 8 is formed into an im- 
age. Reference numeral 22 denotes an area of the 
formed image obtained by image pickup in all directions. 
[0046] Coordinate axes X and Y are set on a plane 
parallel with the image forming plane 1 9 using a focal 
point 20 of the television camera 20 as an origin. In ad- 
dition, a Z axis is set in a direction perpendicular to the 
image forming plane 19 using the focal point 20 of the 
television camera 15 as an origin. The center axis of the 
mirror 14 aligns with the Z axis. In this case, an image 
of the photographing object 1 8 has its direction changed 
by the mirror 14 and passes through the focal point 20 
of the television camera before reaching an image form- 
ing position 21 on the image forming plane 19. 
[0047] Since the mirror 14 is shaped into a hyperbo- 
loid, an extension of a beam directed from the photo- 
graphing object 18 to the mirror 14 always passe 
through a point 23 on the Z axis despite a change in the 
position of the photographing object. Reference numer- 
al 23 is referred to as a "virtual focal point". 
[0048] Thus, the formed image 22 corresponds to an 
image observed from the virtual focal point 23 through 
an azimuth of 360° around the X axis. An elevation di- 
rection at the virtual focal point 23 from an XY plane var- 
ies depending on the angle of view of the television cam- 
era 15. 

[0049] In this manner, images observed through 360° 
around the Z axis in the azimuth direction and at a pre- 
determined elevation direction are formed in the area 
22 of the formed image at one time. The circular formed 
image thus obtained is referred to as an "omnidirectional 
image". 

[0050] An example of an omnidirectional image is 
shown in FIG. 4. 



[0051] Reference numeral 24 denotes an omnidirec- 
tional image that is circular. In this case, when a radial 
direction of the omnidirectional image 24 is denoted by 
R and its circumferential direction is denoted by 6, the 
5 radial direction R corresponds to the direction of the el- 
evation angle of the line of sight from the virtual focal 
point 23 in FIG. 3, and the circumferential direction 6 
corresponds to the direction of the azimuth of the line of 
sight from the virtual focal point 23 in FIG. 3. 
10 [0052] Next, a method for obtaining an image and po- 
sitional information stored in the omnidirectional image 
position storage means 3 and how the image and the 
information are recorded therein will be described with 
reference to FIGS. 5 and 6. 
15 [0053] FIG. 5 is an image showing a traveling path of 
the video sensor device 1 1 . The arrow shows a traveling 
direction, indicating that the device starts from point A 
and moves in the order of A, B, C } D, E, F, C, D, and A. 
Reference numerals 25a, 25b, and 25c denote the po- 
20 sitions of buildings located on the path. 

[0054] While moving along the traveling path shown 
in FIG. 5, the video sensor device 1 1 uses the television 
camera 15 and the GPS device 1 6 mounted thereon to 
obtain an omnidirectional image and positional informa- 
25 tion and then transmits them to the supply center 12. 
The transmission of the omnidirectional image and po- 
sitional information from the video sensor device 11 to 
the supply center 12 can be implemented by using radio 
or wired communication or recording them on a record- 
30 ing medium such as a DVD. 

[0055] FIG. 6 is a schematic view showing how omni- 
directional images and positions are recorded in the 
supply center 12. 

[0056] In FIG. 6, reference numeral 26 denotes an 

35 omnidirectional image position file in which are recorded 
omnidirectional images photographed at the same time 
and their positions which are allocated with record num- 
bers. For explanation, the case where the omnidirec- 
tional image and positional information are recorded to- 

40 gether is shown, but they may be individually recorded 
and have the same record number attached thereto. 
[0057] At an arbitrary point of time after the omnidi- 
rectional image and the positional information have 
been recorded in the omnidirectional image position 

45 storage means 3, the bidirectional communication 
means 10 starts communication between the supply 
center 12 and the terminal device 13 in response to a 
command from the terminal device 13. 
[0058] Operational steps executed by the omnidirec- 

so tional video device after start of communication will be 
explained below with reference to FIGS. 7 to 12. 

(Step 1-1) 

55 [0059] The omnidirectional image position storage 
means 3 has the omnidirectional image position file 26 
recorded therein as shown in FIG. 6. First, positional in- 
formation and a record number are obtained, and all im- 
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age pickup positions are displayed on the image pickup 
position display means 4 of the terminal device 13 via 
the bidirectional communication means 10. 
[0060] FIG. 7 shows all image pickup positions dis- 
played on the image pickup position display means 4. 
Dotted lines in FIG. 7 show the image pickup position. 
[0061] Since the positional information is provided as 
latitude and longitude information obtained by the GPS 
device 16, the image pickup position display means 4 
displays the latitude and longitude information on a re- 
duced scale. Characters A, B, C, D, E, and F in FIG. 7 
indicating positions are for explanation only and are not 
displayed on the image pickup position display means 4. 

(Step 1-2) 

[0062] The user, who has checked all the image pick- 
up positions displayed on the image pickup position dis- 
play means 4 at (step 1 -1), specifies the user's desired 
observation path from all the image pickup positions us- 
ing the view point position indicating means 5. FIG. 8 
shows a result of the indication of the observation path 
using the view point position indicating means 5. This 
figure shows, for example, a result of the indication of a 
path passing through the points A, B, C, and D. In this 
embodiment, the continuous observation path is select- 
ed using the view point position indicating means 5, but 
a discontinuous observation path such as A-B and E-F 
may be selected. 

[0063] Specific examples of the image pickup position 
display means 4 and the view point position indicating 
means 5 will be described. The terminal device 13 rec- 
ognizes all the Image pickup positions and displays 
them on a display screen of the terminal device 13 in 
the format shown in FIG. 7. The user uses as an input 
device a touch panel mounted in such a manner as to 
overlap the display screen, to touch via the touch panel 
the display screen of all the image pickup positions at 
positions corresponding to the observation path, there- 
by inputting the indicated positions. Alternatively, the us- 
er inputs the observation path from a keyboard of the 
terminal device 13. 

(Step 1-3) 

[0064] Record numbers corresponding to the posi- 
tions of the observation path indicated by the view point 
position indicating means 5 at (Step 1-2) are arranged 
in the order of observation and then transmitted to the 
image converting means 6 via the bidirectional commu- 
nication means 10. 

(Step 1-4) 

[0065] Omnidirectional images corresponding to the 
record numbers transmitted from the view point position 
indicating means 5 at (Step 1 -3) are invoked from the 
omnidirectional image position storage means 3 and 



transmitted to the image converting means 6. 
(Step 1-5) 

5 [0066] The omnidirectional images transmitted to the 
image converging means 6 at (Step 1-4) are converted. 
The process method executed by the image converting 
means 6 will be explained with reference to FIGS. 3, 4, 
9, and 10. 

10 [0067] FIG. 9 shows an image into which the omnidi- 
rectional image 24 in FIG. 4 has been converted and 
which is assumed to be observed from the virtual focal 
point 23 shown in FIG. 3. This image is hereafter re- 
ferred to as a "expanded image 27". 
15 [0068] An X direction of the expanded image 27 cor- 
responds to an azimuth direction 6 around the 2 axis 
observed from the virtual focal point 23, thereby allowing 
an image extending through an azimuth of 360° to be 
obtained. In addition, a Y direction of the image corre- 
20 spends to the elevation direction R observed from the 
virtual focal point 23. 

[0069] Accordingly, when an arbitrary point (R1, 91) 
in the omnidirectional image 24 in FIG. 4 is converted 
into a position of coordinates (x1 , y1) in the expanded 
25 image 27, the following equation is established. 

(x1,y1) = F(rt ( ei) (1) 

30 a function F is a coordinate converting function and is 
well known, so detailed description thereof is omitted. 
[0070] At (Step 1-4), the image converting means 6 
invokes the omnidirectional images 24 corresponding to 
the record numbers transmitted from the view point po- 
35 sition indicating means 5 and converts them into the ex- 
panded images 27. The image converting means 6 then 
rearranges the expanded images 27 in the order of the 
observation path specified by the view point position in- 
dicating means 5. 
40 [0071] FIG. 10 shows a sequence of the expanded im- 
ages 27 thus obtained. As shown in this figure, a ex- 
panded image file 28 having the expanded images 27 
arranged in the order of the observation is created and 
transmitted to the image processing means 8 via the bi- 
45 directional communication means 10. 

(Step 1-6) 

[0072] The process method executed by the line-of- 
so sight indicating means 7 and the image processing 
means 8 will be explained with reference to FIGS. 10, 
11, and 12. 

[0073] FIG. 11 shows an example of an image ob- 
tained by the image processing means 8 by cutting a 
55 partial area out from the expanded image 27 in FIG. 9. 
This image is hereafter referred to as a "window image 

29". 

[0074] The window image 29 in FIG. 11 is obtained by 
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specifying a line-of-sight direction, that is, an elevation 
angle and an azimuth and cutting the partial area out 
from the expanded image 27 in FIG. 9. The size of the 
cut-out area corresponds to the angle of view from the 
observation point. 

[0075] (Step 1-6) will be explained below in detail. 
(Step 1-6-(1)) 

[0076] First, the Image processing means 8 uses pre- 
set predetermined elevation angle and azimuth to cre- 
ate a first window image 29 from the expanded image 
file 28 in FIG. 1 0 transmitted via the bidirectional com- 
munication means 10 at (step 1-5), and then displays 
this window image on the image display means 9. FIG. 
12 is a schematic view of the sight indicating means 7. 

(Step 1-6-(2)) 

[0077] Then, the user uses the line-of-sight indicating 
means 7 to change the line-of-sight direction and the 
angle of view for the image displayed on the image dis- 
play means 9. FIG. 12 is a schematic view of the sight 
indicating means 7. 

[0078] In FIG. 12, reference numerals 30a and 30b 
denote elevation angle indicating buttons, reference nu- 
merals 31a and 31b denote azimuth indicating buttons, 
and 32a and 32b denote angle of view indicating but- 
tons. These are displayed on the image display means 
9 and operated by the user using a mouse or the like. 
In addition, they may be implemented by allocating cor- 
responding functions to ten keys of the keyboard. 

(Step 1-6-(3)) 

[0079] Next, the line-of-sight direction and angle of 
view changed by the line-of-sight indicating means 7 are 
transmitted to the image processing means 8, and the 
cut-out area is changed for the next expanded image to 
create the window image 29, which is then displayed on 
the image display means 9. Then, (Step 1-6-(2)) and 
(step 1 -6-(3)) are repeated until all the expanded images 
27 in the expanded image file 28 have been processed. 
[0080] As described above, according to this embod- 
iment, the positions at which the omnidirectional image 
24 was photographed are presented to the user before- 
hand so that an image free from information on unwant- 
ed paths and containing only a required view point po- 
sition can be created and transmitted. Consequently, 
the images of all the photographed areas need not be 
transmitted to thereby reduce the amount of communi- 
cated data and thus communication costs. Additionally, 
the user can display images promptly to reduce the 
amount of waiting time. 

(Embodiment 2) 

[0081] FIGS. 13 and 14 show (Embodiment 2). 
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[0082] FIG. 13 shows an omnidirectional video appa- 
ratus according to (Embodiment 2). The omnidirectional 
image pickup means 1 , the image pickup position ob- 
taining means 2, the omnidirectional image position 

5 storage means 3, and the Image pickup position display 
means 4 are the same as those of (Embodiment 1 ). 
[0083] In (Embodiment 1), the terminal device 13 in- 
puts a desired view point and then processes data re- 
ceived from the supply center 12 depending on its de- 

10 sired line of sight to obtain a target display screen. (Em- 
bodiment 2) differs from (Embodiment 1 ) in that not only 
the desired view point but also a line-of-sight direction 
are specified and communicated to the supply center 1 2 
to obtain the target display screen without the need to 

15 reprocess data returned to the terminal device 13 from 
the supply center 12. 

[0084] A line-of-sight vector indicating means 33 pro- 
vided in the terminal device 13 similarly to the image 
pickup position display means 4 specifies user's desired 

20 view point position and line-of-sight direction for an im- 
age pickup position displayed by the image pickup po- 
sition display means 4. An output from the line-of-sight 
vector indicating means 33 is input to an image extract- 
ing and converting means 34 via the bidirectional com- 

25 munication means 10. 

[0085] The image extracting and converting means 
34 provided in the supply center 12 invokes the omnidi- 
rectional image 24 from the omnidirectional image po- 
sition file 26 of the omnidirectional image position stor- 

30 age means 3, the omnidirectional image 24 having been 
recorded at the view point position indicated by the line- 
of-sight vector indicating means 33, The image extract- 
ing and converting means 34 then generates the win- 
dow image 29 photographed in the line-of-sight direction 

35 Indicated by the line-of-sight vector indicating means 33. 
[0086] A sequence of the window images 29 created 
by the image extracting and converting means 34 are 
compressed by the image compressing means 35 and 
transmitted via the bidirectional communication means 

40 1 o to the terminal device 1 3, where an image recovering 
means 36 recovers the signal transmitted from the im- 
age compressing means 35. 

[0087] The sequence of images recovered by the im- 
age recovering means 36 are displayed on the image 

45 display means 9. 

[0088] (Embodiment 2) differs from (Embodiment 1) 
in that the supply center 12 has the omnidirectional im- 
age position storage means 3, the image extracting and 
converting means 34, and the image compressing 

so means 35 and in that the terminal device 1 3 has the im- 
age pickup position display means 4, the line-of-sight 
vector indicating means 33, the image recovering 
means 36, and the image display means 9. 
[0089] The operational steps executed by the omnidi- 

55 rectional video apparatus after communication has 
been started will be explained. 
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(Step 2-1) 

[0090] The omnidirectional image position storage 
means 3 has the omnidirectional image position file 26 
shown in FIG. 6, recorded therein. First, positional infor- 
mation and record numbers are taken out from the stor- 
age means, and the image pickup positions are dis- 
played on the image pickup position display means 4 of 
the terminal device 13 via the bidirectional communica- 
tion means 1 0. The image pickup positions displayed by 
the image pickup position display means 4 are similar 
to those in FIG. 7. 

(Step 2-2) 

[0091] On checking the image pickup positions dis- 
played on the image pickup position display means 4 at 
(step 2-1), the user specifies his or her desired obser- 
vation path and line-of-sight directions for each obser- 
vation path, from all the image pickup positions, using 
the line-of-sight vector indicating means 33. FIG. 14 
shows the observation path and line-of-sight directions 
indicated using the line-of-sight vector indicating means 
33. The observation path is shown by solid arrows 41 , 
and the line-of-sight directions are denoted by thick 
white arrows 42. Specifically, the indication of the ob- 
servation path is input from the display screen or key- 
board (not shown) of the terminal device 1 3 in the same 
manner as in (Embodiment 1). 

(Step 2-3) 

[0092] The observation path and Hne-of-sight direc- 
tions indicated by the line-of-sight vector indicating 
means 33, that is, the elevation angle and the azimuth 
are transmitted to the image extracting and converting 
means 34 via the bidirectional communication means 
10. 

(Step 2-4) 

[0093] The image extracting and converting means 
34 invokes the omnidirectional image 24 from the om- 
nidirectional image position storage means 3, the omni- 
directionaJ image 24 corresponding to the observation 
path transmitted from the line-of-sight vector indicating 
means 33. 

(Step 2-5) 

[0094] The image extracting and converting means 
34 uses the omnidirectional image 24 obtained at (Step 
2-4) and the line-of-sight directions obtained at (Step 
2-3) to create the window images 29 shown in FIG. 11 . 
In this case, the expanded images 27 shown in FIG. 9 
need not necessarily be created but the window images 
29 are desirably created directly from the omnidirection- 
al images 24. 



[0095] The created window images 29 are rearranged 
along the observation path and then transmitted to the 
image compressing means 35. 

5 (Step 2-6) 

[0096] The image compressing means 35 compress- 
es the sequence of the window images 29 obtained at 
(Step 2-5) to reduce the amount of signals. Since the 

10 window images 29 have been rearranged in the order 
of the observation path, for example, the order of dis- 
play, the image compression may be video compres- 
sion, for example, that based on the MPEG. The com- 
pressed signal is transmitted to the image recovering 

15 means 36 of the terminal device 13 via the bidirectional 
communication means 10. 

(Step 2-7) 

20 [0097] The image recovering means 36 stores the im- 
ages transmitted from the image compressing means 
35 in a primary buffer (not shown) and recovers and 
transmits them to the image display means 9 for display. 
[0098] Thus, according to (Embodiment 2), only im- 

*s ages required for display can be transmitted by previ- 
ously presenting the user the positions at which the om- 
nidirectional images 24 were photographed so that the 
user can indicate required view point position and line- 
of-sight direction which are free from information on un- 

30 wanted paths. 

[0099] Further, the images are compressed before 
transmission to reduce the amount of communicated 
data. 

[0100] In addition, the user (the terminal device 13) 
55 must execute only the process of recovering the images 
and need not execute conversions based on an image 
converting process, so that the images are displayed at 
equal time intervals to provide the user with the feeling 
of being at the site without compromising smooth dis- 
*o play or reproduction. 

(Embodiment 3) 

[0101] FIGS. 15 to 17 show (Embodiment 3). 

45 [0102] FIG. 15 shows an omnidirectional video appa- 
ratus according to (Embodiment 3). 
[0103] (Embodiment 3) is the same as (Embodiment 
2) except that the image pickup position display means 
4 of the terminal device 13 is replaced with a map data 

so storage means 37 and a synthesizing position display 
means 38. 

[0104] The map data storage means 37 according to 
(Embodiment 3) has map data for an area photographed 
by the omnidirectional image pickup means 1 , previous- 
55 ly written thereto. The synthesizing position display 
means 38 is connected to the omnidirectional image po- 
sition storage means 3 of the supply center 12 via the 
bidirectional communication means 10 to display the 
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photographing positions stored in the omnidirectional 
image position storage means 3 as in (Embodiment 1) 
and (Embodiment 2), while synthesizing and displaying 
the map information stored in the map data storage 
means 37, on the same screen. 
[01 05] FIG. 1 6 shows a display format for the synthe- 
sizing position display means 38. As also shown in FIG. 
5, reference numerals 25a, 25b, and 25c denote the po- 
sitions of the buildings on the path. In this manner, not 
only the traveling path is displayed but also the map in- 
formation such as the locations of buildings are added 
and displayed based on the map data. 
[0106] FIG. 17 shows an observation path and a line- 
of-sight direction indicated by the view point vector in- 
dicating means 33. The arrows 41 in FIG. 1 7 denote the 
observation path, and the thick white arrows 42 denote 
the line-of-sight directions. If, for example, the user de- 
sires to observe the buildings 25a, 25b, and 25c, since 
their positions have been confirmed in advance, the us- 
er selects an observation path passing through the 
points A, B, C, and F and indicates the line-of-sight di- 
rections toward the buildings 25a, 25b, and 25c. 
[0107] According to (Embodiment 3), the positions 
where the omnidirectional images were photographed 
and the map information are synthesized and displayed 
to enable the user to recognize environments of the pho- 
tographing positions beforehand to accurately indicate 
an observation path and a line-of-sight direction de- 
pending on the user's tastes. 

[0108] In this embodiment, the map data storage 
means 37 is provided inside the terminal device 13, but 
similar effects are clearly obtained by providing it inside 
the supply center 12 or Invoking the map data storage 
means 37 installed in another device (not shown) con- 
nected to the bidirectional communication device. 

(Embodiment 4) 

[0109] FIGS. 1 8 to 20 shows (Embodiment 4). 
[0110] (Embodiment 4) is the same as (Embodiment 
1) except that the map data storage means 37 and the 
synthesizing position display means 38 replace the im- 
age pickup position display means 4 according to (Em- 
bodiment 1) shown in FIG. 1 and that an object specify- 
ing means 39 and a view point position computing 
means 40 replace the view point position indicating 
means 5. The map data storage means 37 and the syn- 
thesizing position display means 38 are the same as 
those in (Embodiments). 

[01 1 1 ] The object specifying means 39 is an input de- 
vice for specifying only a desired object of the objects 
on all the image pickup paths displayed by the synthe- 
sizing position display means 38. 
[0112] A view point position computing means 40 
computes an optimal path from the position of the object 
specified by the object specifying means 39. 



(Step 4-1) 

[0113] On receiving positional information and record 
numbers from the omnidirectional image position stor- 

a age means 3 via the bidirectional communication means 
10, the synthesizing position display means 38 invokes 
from the map data storage means 37 map data for an 
area photographed by the omnidirectional image pickup 
means 1 , and synthesizes and displays them. 

10 [0114] A path diagram displayed by the synthesizing 
position display means 38 and having the additional 
map data is similar to that in FIG. 16. As also shown in 
FIG. 5, reference numerals 25a, 25b, and 25c denote 
the positions of the buildings on the path. In this manner, 

15 not only the traveling path is displayed but also the map 
information such as the locations of buildings are added 
and displayed based on the map data. 



20 



(Step 4-2-(1)) 



[0115] On checking all the photographing positions 
displayed on the synthesizing position display means 38 
at (Step 4-1 ), the user uses the object specifying means 
39 to specify only a desired object on the path of the 
25 photographing positions displayed on the synthesizing 
position display means 38. 

[0116] FIG. 19 shows a result of specification of a plu- 
rality of objects by means of the object specifying means 
39. The thick white arrows denote the specified objects; 
30 in FIG. 1 9, the buildings 25a and 25c have been speci- 
fied. 

(Step 4-2-(2)) 

35 [0117] The view point position computing means 40 
in FIG, 18 automatically computes a path for observing 
the specified buildings 25a and 25c as shown in FIG. 19. 
[0118] FIG. 20 shows a result of display of the result 
of the computation of the observation path. The path is 

40 shown by an arrow 44. To compute the path, for exam- 
ple, all paths joining the buildings 25a and 25c in FIG. 
19 together and the shortest of them is determined as 
the observation path. A different method may be used 
to compute the path as long as it can compute a path 

45 between a plurality of objects. 

[0119] Since the observation path has been estab- 
lished at (Step 4-2-(2)), the subsequent steps are similar 
to (step 1-3) to (Step 1-6) in (Embodiment 1). 
[0120] According to (Embodiment 4), the positions 

so where the omnidirectional images 24 were photo- 
graphed and the map information are synthesized and 
displayed to enable the user to recognize environments 
of the photographing positions beforehand to accurately 
indicate an observation path depending on the user's 

55 tastes. In addition, the path can be automatically created 
simply by specifying the user's desired place, so that 
images of the user's desired place can be simply repro- 
duced without the need to Input the view point position. 
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[0121] In this embodiment, the observation path is au- 
tomatically computed while the line-of-sight direction is 
indicated by the user, but the direction of the object rel- 
ative to the observation path can be easily estimated 
from this path. Thus, the view point position computing 
means 40 can be used to mathematically obtain the ob- 
servation path and the line-of-sight direction. In that 
case, the process will be similar to (Step 2-3) and (Step 
2-7) in (Embodiment 2). 

[0122] Specifically, the terminal device 13 has an 
computation means for computing an optimal observa- 
tion path joining each view point with the position of the 
object specified by the object specifying means 39 to 
output view point information and estimating the direc- 
tion of the object relative to the observation path to out- 
put line-of-sight information. The supply center 12 has 
the image extracting and converting means 34 for invok- 
ing required image data from the omnidirectional image 
position storage means 3 to convert them into expanded 
images based on the view point information and line-of- 
sight information indicated by the computation means 
via the bidirectional communication means 10, and 
processing the expanded images 27 depending on the 
line-of-sight information. The terminal device 1 3 has the 
image display means 9 for displaying as an image a sig- 
nal received from the image extracting and converting 
means 34 via the bidirectional communication means 
10. 



Claims 

1. An omnidirectional video output method, compris- 
ing: 

photographing, at a view point position moving 
within a service area, images around said view 
point position and recording the images as an 
omnidirectional image position file (26) in such 
a manner that the images correspond to image 
pickup position data on positions where the im- 
ages were picked up; 

presenting a user the image pickup position da- 
ta on positions where omnidirectional images 
were recorded; 

invoking required image data from said omnidi- 
rectional image position file (26) based on in- 
formation on a viewpoint in said service area, 
the information being specified by the user; and 
processing each invoked image data based on 
a line-of-sight direction (42) specified by the us- 
er and outputting an image. 

2. An omnidirectional vide output method, comprising: 

when a terminal device (13) communicate with 
a supply center (12) and receives required im- 
age data from said supply center (12) to output 



images, 

presenting said terminal device (13) image 
pickup position data from a supply center (12) 
having an omnidirectional image position file 

5 (26) comprising images photographed at each 

view point position in all directions by moving 
within a service area as well as data on image 
pickup positions at which the images were 
picked up, the image pickup position data con- 

10 taining data on positions where the images in 

all directions were picked up; 
transmitting information on a desired view point 
on the presented traveling path to the supply 
center (12); 

15 based on the view point information specified 

by the terminal device (13), the supply center 

(12) invoking required image data from an om- 
nidirectional image position file (26) to convert 
them into expanded images (27) and transmit- 

20 ting the images to the terminal device (13); and 

the terminal device (13) processing the re- 
ceived expanded images (27) depending on a 
desired line-of-sight direction (42) to output im- 
ages. 

25 

3. An omnidirectional vide output method, comprising: 

when a terminal device (13) communicate with 
a supply center (12) and receives required im- 
30 age data from said supply center (12) to output 

images, 

presenting said terminal device (13) image 
pickup position data from a supply center (12) 
having an omnidirectional image position file 

35 (26) comprising images photographed at each 

view point position in all directions by moving 
within a service area as well as data on image 
pickup positions at which the Images were 
picked up, the image pickup position data con- 

40 taining data on positions where the images in 

all directions were picked up; 
transmitting information on a desired view point 
and a desired line of sight on the presented 
traveling path to the supply center (12); 

45 based on the view point information, the supply 

center (12) invoking required image data from 
an omnidirectional image position file (26) to 
process them based on said llne-of-sight infor- 
mation specified by the terminal device (13) to 

so convert them into window images (29) and 

transmitting the images to the terminal device 

(13) ; and 

the terminal device (13) outputting images. 

55 4. The omnidirectional video output method according 
to Claim 3, wherein when the image pickup position 
data on positions where omnidirectional images 
were recorded are presented from the supply center 
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(1 2) to the terminal device (1 3), a map of the service 
area and said image pickup position data are syn- 
thesized and displayed. 

The omnidirectional video output method according 5 
to Claim 2, wherein when the image pickup position 
data on positions where omnidirectional images 
were recorded are presented from the supply center 
(12) to the terminal device (13), 

10 

the terminal device (13) synthesizes and dis- 
plays a map of the service area and said image 
pickup position data, 

the terminal device (13) computes an optimal 
path joining each view point with a position of 19 
a specified one of objects on ail displayed im- 
age pickup paths, 

based on a result of the computation, the supply 
center (12) invokes required image data from 
the omnidirectional image position file (26) to 20 
convert them into the expanded images (27) 
and transmits the images to the terminal device 
(13), and 

on receiving the expanded images (27) at the 
desired view point position, the terminal device 25 
(13) processes the received expanded images 
(27) depending on the desired line-of-sight di- 
rection (42) to output images. 

The omnidirectional video output method according 30 
to Claim 2, wherein when the image pickup position 
data on positions where omnidirectional images 
were recorded are presented from the supply center 
(12) to the terminal device (13), 

35 

the terminal device (13) synthesizes and dis- 
plays a map of the service area and said image 
pickup position data, 

the terminal device (13) computes an optimal 
path joining each view point with a position of ^ 
a specified one of objects on all displayed im- 
age pickup paths, as well as the line-of-sight 
direction (42) of the object relative to the obser- 
vation path (41 ), and transmits the optimal path 
and the line-of-sight direction to the supply 45 
center (12), 

based on the observation path (41) of a result 
of the computation, the supply center (12) in- 
vokes required Image data from the omnidirec- 
tional image position file (26) to process them so 
based on said iine-of-sight information of the 
result of the computation to convert them into 
window images (29) and transmits the images 
to the terminal device (13), and 
the terminal device (13) outputs images. 55 

An omnidirectional vide output apparatus in which 
a terminal device (13) communicate with a supply 



center (12) and receives required image data from 
said supply center (12) to output images, character- 
ized in that: 

the supply cenier (12) has omnidirectional im- 
age position storage means (3) having an om- 
nidirectional image position file (26) comprising 
images photographed at each view point posi- 
tion in all directions by moving within a service 
area as well as data on image pickup positions 
at which the images were picked up, the image 
pickup position data containing data on posi- 
tions where the images were picked up; 
the apparatus includes bidirectional communi- 
cation means (10) for transmitting and receiv- 
ing information between the supply center (12) 
and the terminal device (13); 
the terminal device (13) has image pickup po- 
sition display means (4) for displaying image 
pickup position data on positions where omni- 
directional images were photographed, the da- 
ta being read out from said omnidirectional im- 
age position storage means (3) via said bidirec- 
tional communication means (10); 
the terminal device (13) has view point position 
indicating means (5) for outputting information 
on a desired view point on a traveling path 
based on a display by said image pickup posi- 
tion display means (4); 

the supply center (12) has image converting 
means (6) for invoking required image data 
from the omnidirectional image position stor- 
age means (3) to convert them into expanded 
images (27) based on the view point informa- 
tion indicated by said view point position indi- 
cating means (5) via said bidirectional commu- 
nication means (10); and 
the terminal device (13) has image processing 
means (8) for processing the expanded images 
(27) received from the image converting means 
(6) via said bidirectional communication means 
(1 0), depending on a desired line-of-sight direc- 
tion (42), and image display means (9) for dis- 
playing window images (29) output by the im- 
age processing means (8). 

8. An omnidirectional vide output apparatus in which 
a terminal device (13) communicate with a supply 
center (1 2) and receives required image data from 
said supply center (12) to output images, character- 
ized in that 

the supply center (12) has omnidirectional im- 
age position storage means (3) having an om- 
nidirectional image position file (26) comprising 
images photographed at each view point posi- 
tion in ail directions by moving within a service 
area as well as data on image pickup positions 
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at which the images were picked up, the image 
pickup position data containing data on posi- 
tions where the images were picked up; 
the apparatus includes bidirectional communi- 
cation means (10) for transmitting and receiv- 5 
ing information between the supply center (12) 
and the terminal device (13); 
the terminal device (13) has image pickup po- 
sition display means (4) for displaying image 
pickup position data on positions where omni- io 
directional images were photographed, the da- 
ta being read out from said omnidirectional im- 
age position storage means (3) via said bidirec- 
tional communication means (10); 
the terminal device (1 3) has line-of-sight vector 15 
indicating means (33) for outputting information 
on a desired view point and a desired line of 
sight on a traveling path based on a display by 
said image pickup position display means (4); 
the supply center (1 2) has image extracting and 20 
converting means (34) for invoking required im- 
age data from the omnidirectional image posi- 
tion storage means (3) based on the view point 
information received from said line-of-sight 
vector indicating means (33) via said bidirec- 25 
tional communication means (10) and process- 
ing and outputting the data based on the line- 
of-sight information received from said line-of- 
sight vector indicating means (33); and 
the terminal device (13) has image display 30 
means (9) for displaying as an image a signal 
received from the supply center (12) via the bi- 
directional communication means (10). 

9. An omnidirectional vide output apparatus in which 35 
a terminal device (13) communicate with a supply 
center (12) and receives required image data from 
said supply center (1 2) to output images, character- 
ized in that: 

40 

the supply center (12) has omnidirectional im- 
age position storage means (3) having an om- 
nidirectional image position file (26) comprising 
images photographed at each view point posi- 
tion in all directions by moving within a service 45 
area as well as data on image pickup positions 
at which the images were picked up, the image 
pickup position data containing data on posi- 
tions where the images were picked up; 
the apparatus includes bidirectional communi- so 
cation means (10) for transmitting and receiv- 
ing information between the supply center (12) 
and the terminal device (13); 
the terminal device (13) has map data storage 
means (37) having a map of the service area ss 
registered therein; 

the terminal device (1 3) has synthesizing posi- 
tion display means (38) for displaying the map 



read out from said map data storage means 
(37) and the image pickup position data on the 
positions where omnidirectional images were 
photographed, the data having been read out 
from said omnidirectional image positions stor- 
age means (3) via said bidirectional communi- 
cation means (10), in such a manner that the 
map and the data overlap each other; 
the terminal device (1 3) has line-of-sight vector 
indicating means (33) for outputting information 
on a desired view point and a desired line of 
sight on a traveling path based on a display by 
said synthesizing position display means (38); 
the supply center (12) has image extracting and 
converting means (34) for invoking required im- 
age data from the omnidirectional image posi- 
tion storage means (3) based on the view point 
information received from said line-of-sight 
vector indicating means (33) via said bidirec- 
tional communication means (1 0) and process- 
ing and converting the data Into window images 
(29) based on the line-of-sight information re- 
ceived from said line-of-sight vector indicating 
means (33); and 

the terminal device (13) has image display 
means (9) for displaying the window images 
(29) received from the supply center (12) via 
the bidirectional communication means (10). 

10. An omnidirectional vide output apparatus in which 
a terminal device (13) communicate with a supply 
center (12) and receives required Image data from 
said supply center (12) to output images, character- 
ized in that: 

the supply center (12) has omnidirectional im- 
age position storage means (3) having an om- 
nidirectional image position file (26) comprising 
images photographed at each view point posi- 
tion in all directions by moving within a service 
area as well as data on image pickup positions 
at which the Images were picked up, the image 
pickup position data containing data on posi- 
tions where the images were picked up; 
the apparatus includes bidirectional communi- 
cation means (10) for transmitting and receiv- 
ing information between the supply center (12) 
and the terminal device (13); 
the terminal device (13) has map data storage 
means (37) having a map of the service area 
registered therein; 

the terminal device (13) has synthesizing posi- 
tion display means (38) for displaying the map 
read out from said map data storage means 
(37) and the image pickup position data on the 
positions where omnidirectional images were 
photographed, the data having been read out 
from said omnidirectional image positions stor- 
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age means (3) via said bidirectional communi- 
cation means (10), in such a manner that the 
map and the data overlap each other; 
the terminal device (13) has object specifying 
means (39) for specifying a specified one of ob- * 
jects on all image pickup paths displayed on the 
synthesizing position display means (38); the 
terminal device (13) has view point position 
computing means (40) for computing an opti- 
mal path joining each view point with a position 10 
of the object (39) specified by said object spec- 
ifying means (39); 

the supply center (12) has image converting 
means (6) for invoking required image data 
from the omnidirectional image position stor- 15 
age means (3) based on the view point infor- 
mation indicated by said view point position 
computing means (40) via said bidirectional 
communication means (10) and converting the 
data into expanded images (27); and 20 
the terminal device (13) has image processing 
means (8) for processing the expanded images 
(27) received from the image converting means 
(6) via said bidirectional communication means 
(1 0), depending on a desired line-of-sight direc- 25 
tion, and image display means (9) for displaying 
window images (29) output by the image 
processing means (8). 

11. An omnidirectional vide output apparatus in which 30 
a terminal device (13) communicate with a supply 
center (12) and receives required image data from 
said supply center (12) to output images, the appa- 
ratus being characterized in that: 

35 

the supply center (12) has omnidirectional im- 
age position storage means (3) having an om- 
nidirectional image position file (26) comprising 
images photographed at each view point posi- 
tion in ail directions by moving within a service *o 
area as well as data on image pickup positions 
at which the images were picked up, the image 
pickup position data containing data on posi- 
tions where the images were picked up; 
the apparatus includes bidirectional communi- 45 
cation means (10) for transmitting and receiv- 
ing information between the supply center (12) 
and the terminal device (13); 
the terminal device (13) has map datastorage 
means (37) having a map of the service area so 
registered therein; 

the terminal device (13) has synthesizing posi- 
tion display means (38) for displaying the map 
read out from said map data storage means 
(37) and the image pickup position data on the ss 
positions where omnidirectional images were 
photographed, the data having been read out 
from said omnidirectional image positions stor- 



age means (3) via said bidirectional communi- 
cation means (10), in such a manner that the 
map and the data overlap each other; 
the terminal device (13) has object specifying 
means (39) for specifying a specified one of ob- 
jects on all image pickup paths displayed on the 
synthesizing position display means (38); 
the terminal device (13) has computation 
means for computing an optimal path (44) join- 
ing each view point with a position of the object 
specified by said object specifying means (39), 
to output view point information, and estimating 
a direction of the object relative to said obser- 
vation path (44) to output line-of-sight informa- 
tion; 

the supply center (12) has image converting 
means (6) for invoking required image data 
from the omnidirectional image position stor- 
age means (3) based on the view point infor- 
mation and the line-of-sight information indicat- 
ed by said computation means via said bidirec- 
tional communication means (10), converting 
the data into expanded images (27), and 
processing the expanded images (27) depend- 
ing on said line-of-sight information; and 
the terminal device (13) has image display 
means (9) for displaying as an image a signal 
received from the image converting means (6) 
via said bidirectional communication means 
(10). 
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